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Abstract. Stereoscopic visualization adds an additional dimension to the 
viewer's experience, giving them a sense of distance. In a general relativistic 
visualization, distance can be measured in a variety of ways. We argue that the 
affine distance, which matches the usual notion of distance in flat spacetime, is 
a natural distance to use in curved spacetime. As an example, we apply affine 
distance to the visualization of the interior of a black hole. Affine distance is 
not the distance perceived with normal binocular vision in curved spacetime. 
However, the failure of binocular vision is simply a limitation of animals who 
have evolved in flat spacetime, not a fundamental obstacle to depth perception in 
curved spacetime. Trinocular vision would provide superior depth perception. 
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1. Introduction 

Visualization has the potential to make general relativity accessible to non-experts, 
and to give experts deeper insights into the theory [U [2] . Presenting depth information 
stereoscopicalljH adds an extra dimension to the viewer's experience. The depth 
information has the potential to help the viewer make better sense of a highly curved 
spacetime, such as that near and inside a black hole. As is common in relativity, 
what really happens often fails to conform to expectation drawn from slow-moving 
experience in flat spacetime. Once a viewer is able to overcome preconceptions, 
distance information can offer deeper insight. 

As demonstrated in ^ normal binocular vision fails in curved spacetime. 
Binocular distance measures the radius of curvature of wavefronts of light from an 
emitter. In curved spacetime, wavefronts become locally ellipsoidal rather than 
spherical, with two distinct radii of curvature in orthogonal directions. The conflicting 
binocular distances would confuse a human being. In extreme situations. Figure [l] 
the scenes seen by two eyes a small distance apart can differ so much as to confound 
binocular vision entirely. 

While binocular vision fails, stereoscopic views of curved spacetime can 
nevertheless be created for human consumption by constructing effective 3-dimensional 
scenes in which objects are placed at their apparent angular positions and at suitably 
chosen distances. In [J3]we argue that the most natural measure of distance in general 
relativistic visualization is the affine distance. The affine distance coincides with 
binocular distance in flat spacetime, but differs in curved spacetime. 

In Sj4]we use affine distance to construct stereoscopic visualizations near and inside 
a Schwarzschild black hole. 

The affine distance measures the distance to where an object used to be at the 
time it emitted its light. This distance is not directly measurable by an observer using 
only local measurements, no more than an astronomer can reach out and touch a 
quasar. In Sj5]we discuss perceptual distances that an observer could use to estimate 
the affine distance. While binocular vision does not give the viewer a consistent sense 
of distance in curved spacetime, trinocular vision performs better, as discussed in §5.31 

All the visualizations in this paper were created with the Black Hole Flight 
Simulator [3j. 

2. Binocular vision fails in curved spacetime 

Figure [T] illustrates the failure of binocular vision in curved spacetime. The Figure 
shows the Earth behind a Schwarzschild black hole [S] . 

The top panel of Figure [T] shows the binocular view, as seen by two eyes a modest 
distance apart. In normal flat space the images of distant objects seen by two eyes 
are shifted slightly, and the brain uses the slight difference to infer a distance. In 
curved space the images of distant objects may be altered in size and shape enough 
to confound binocular vision. In Figure [1] both the Earth behind the black hole, and 
the distant Galaxy, show discordant image pairs that defeat binocular interpretation. 
Whereas the right eye (left image) sees the Earth form a ring around the black hole, 
the left eye (right image) sees the Earth as two disconnected crescents. 

I "Stereoscopic" refers to image pairs that encode distance information. In curved space-time there 
are various ways to measure distance, and any one of these could be encoded in the stereoscopic view. 
"Binocular" refers specifically to the actual view seen with two eyes. 
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Figure 1. (Top) the binocular view, eompared to (middle) the stereo view using 
the affine distance, and (bottom) the binocular/stereo view with lensing turned 
off, of the Earth behind a Schwarzsehild black hole, observed by an observer at rest 
from a distance of 5 Schwarzsehild radii. The binocular view at top is confusing 
because the image pair fails to mesh in the fashion that binocular vision expects. 
The background is Axel Mellinger's Milky Way 4 (with permission). To view, 
cross your eyes, and relax your focus. 
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For comparison, the middle panel of Figure [T] shows a stereo view of the same 
scene, in which objects have been put at their afhne distances in an effective 3- 
dimensional construction of the scene, as advocated in this paper, fJ31 The bottom 
panel shows the familiar binocular/stereo view of the same scene with gravitational 
lensing by the black hole turned off. Among other things, gravitational lensing 
magnifies the apparent size of the black hole: a distant observer sees a Schwarzschild 
black hole 3a/3/2 w 2.6 times larger than its Schwarzschild radius. 

In the scene illustrated in Figure [1] the observer is at rest at a radius of 5 
Schwarzshild radii from the black hole (a Schwarzschild radius is the radius of the 
black hole's horizon). The field of view is 105° across the diagonal (approximately 
74° X 74°). The eyes are separated by 0.2 Schwarzschild radii. Grid lines on the 
black hole are aligned with Galactic coordinates, and are every 30° in latitude and 
longitude. The observer is viewing towards Galactic longitude and latitude I = 263.6°, 
b = —16.3°. The Earth has a radius of 1/2 a Schwarzschild radius, and is located at a 
radius of 3 Schwarzschild radii from the black hole. Details of how the Earth is placed 
as if it were a rigid body in the curved spacetime are described in [Appendix A The 
configuration is not realistic: if the Earth were this close to a black hole, it would be 
tidally torn apart in moments. 

An animated version of Figure [U showing the Earth in orbit around a black hole, 
is at [5], in both normal and stereo representations. 



3. Affine distance 



The perceived distance to an object would ideally match the distance measured with 
a tape measure. In relativity however, defining a suitable tape measure presents 
challenges. The first challenge is that the apparent length of a tape measure depends 
on its velocity relative to the observer. To make measurements that are useful to the 
observer, the tape measure should be at rest relative to the observer. 

The second challenge is to select the correct path along which to measure the 
distance to an object. This is the path traveled by the light, from where it left the 
object to where it arrives at the observer. 

The third challenge is to choose the velocity of the tape measure at all points along 
the light path from emitter to observer. In the flat spacetime of special relativity, the 
tape measure can be taken to be everywhere at rest with respect to the global rest 
frame. In curved spacetime, however, there is no global rest frame. The most natural 
choice is to require that, while the tape measure necessarily moves relative to the light, 
it should not accelerate relative to the light, that is, the light should not undergo any 
redshift or blueshift as it moves from point to point along the tape measure. 

In summary, the perceived distance that should be encoded in the stereoscopic 
visualization is the distance traveled by the light as measured by a tape measure that 
never accelerates relative to the light and is moving with the observer's velocity at the 
time of observation. While this prescription sounds complicated, it is well-known in 
general relativity, and is called the affine distance [61 p. 575]. 

The affine distance is the affine parameter A normalized in a certain way, as we will 
now discuss. Mathematically, an interval dA of affine parameter A along the trajectory 
of a massless particle (a light ray) is defined so that the derivative of the coordinates 
along the path with respect to affine parameter A equals the 4-momentum p^^ of 
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Figure 2. In special relativity, the scene seen by an observer moving tiirougii 
the scene (right) is relativistically beamed compared to the scene seen by an 
observer at rest relative to the scene (left). On the left, the observer at the 
center of the circle is at rest relative to the surrounding scene. On the right, 
the observer is moving to the right through the same scene at v = 0.6 times 
the speed of light. The arrowed lines represent energy-momenta of photons. 
The length of an arrowed line is proportional to the afHne distance, which is 
proportional to the perceived energy of the photon. The scene ahead of the 
moving observer appears concentrated, blueshifted, and farther away, while the 
scene behind appears expanded, redshifted, and closer. 



the particle: 

The afhne parameter defined by equation ([1} is defined up to an arbitrary 
normalization factor. The affine distance is defined to be the affine parameter 
normalized such that it measures proper distances in the local rest frame of the 
observer. Normalizing the afhne distance in this fashion is equivalent to normalizing 
the afhne parameter such that the photon momentum has unit energy as perceived 
by the observer. It is convenient to denote this normalized photon momentum by the 
photon wavevector k^, 

By construction, the wavevector is normalized to unit energy, k* = E ~ 1, at the point 
of observation by, and in the rest frame of, the observer. With this convention, the 
affine distance A has units of length, while the photon wavevector fc^ is dimensionless. 



3.1. Affine distance in special relativity 

In the flat (Minkowski) spacetime of special relativity, the affine distance to any object 
coincides with the distance measured with a tape measure in the observer's rest frame. 
The affine distance also coincides with both the size and binocular distances described 
in ^ Thus in flat spacetime all the usual rules of 3D stereo rendering and perception 
apply. 

If two observers at the same point are moving relative to each other, then they 
measure a different affine distance along the same light ray. Each observer regards the 
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Figure 3. Coordinate system. The emitter and observer are separated by angle 
in a polar coordinate system {r, 6, (f)} about the black hole. A light ray from 
emitter to observer subtends apparent angle x from the vertical axis. 

affine distance as measuring the number of wavelengths along the path of the photon. 
The number of wavelengths is proportional to the frequency, or energy, of the photon. 
Thus the afhne distance A to an emitter measured by an observer is proportional to 
the perceived energy E of the photon 

XozE . (3) 

Figure [5] illustrates how this works. On the left side of the figure, the observer 
(unfilled circle) at the center is at rest relative the surrounding scene. The stars 
represent parts of the scene. Two sets of stars are shown, one at distance 0.5, the other 
at distance 1 from the observer. On the right side of the figure, the observer (unfilled 
circle) is moving through the same scene to the right at velocity v = 0.6 times the speed 
of light. For both stationary and moving observers, the arrowed lines converging on 
the observer represent energy-momenta of photons emitted by the stars. The energy- 
momenta of photons are null 4- vectors. The photon energy-momenta perceived by the 
two observers are Lorentz-boosted relative to each other. If, relative to the observer 
at rest, the stars lie on a celestial sphere of radius 1, then, relative to the moving 
observer, the same stars lie on a celestial ellipsoid that is stretched by the Lorentz 
gamma factor 7 in the direction of motion, and the observer is displaced a distance 7U 
from the center of the ellipsoid to a focus of the ellipsoid. The Lorentz boost produces 
the relativistic aberration and red/blue-shifting illustrated in the figure. The moving 
observer sees the scene ahead to be concentrated, blueshifted, and farther away, while 
the scene behind is expanded, redshifted, and closer. 

3.2. Affine distance in a Schwarzschild black hole 

This section provides some techical details of ray-tracing in the Schwarzschild [71 [S] 
geometry, used to construct the visualizations illustrated in 21 Reading this section 
is not necessary to appreciate the visualizations. 

A Schwarzschild black hole is the simplest kind of black hole, a static, spherically 
symmetric black hole, characterized entirely by its mass M. The Schwarzschild metric 
in polar coordinates x'^ = {t,r,d,(f)}, Figure [31 is, in geometric units (c = G = 1), 

ds^ ^~Bdt^ +B-^dr^ +r^{de^ +sm'^ed(j)'^) , (4) 
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where 

B=l- 2M/r . (5) 

The vanishing of B defines the horizon, the Schwarzschild radius rg = 2M. In the 
Schwarzschild metric, the coordinate 4- velocity k'^ = dx'^/dX along a null geodesic 
satisfies three conservation laws, associated with energy, mass, and angular momentum 
J per unit energy, 

fc* = l/B, k' = ±^/]~~my^ , k^ = J/r^. (6) 

Here the photon energy fc* has been normalized to one as perceived by observers at 
rest at infinity. Integrating dr/dA = k^ yields the affine distance A between emitter 
and observer, normalized to observers at rest at infinity: 

A = / , . (7) 

iron, ^/T^^^J^ 

The observed affine distance is then 

Aobs = ^^obsA (8) 

where E^hs is the photon energy perceived by the observer relative to the photon 
energy perceived by observers at rest at infinity. If the observer has specific energy E 
and angular momentum L, then their coordinate 4- velocity m'' = dx^/dr is 



E 



= - , vT ^ ±y^E^ - B{l + L^/r^) , = L/r^ . (9) 



B 

The observed photon energy ii^obs = — W/ifc'' is then 



EtV{1- Bjyr^) [E^ -B{1+ Uy)] L J ^^^^ 
Eohs ^ — ■ (10) 

The view from inside a black hole is most symmetrical in the frame of an observer 
radially free-falling on the zero-energy geodesic, E = 0, L = 0, a.t the border between 
ingoing {E > 0) and outgoing (_E < 0). An observer on the zero-energy radial geodesic 
sees a photon of angular momentum J subtend an angle x away from the vertical axis 
(see Figure [S]) given by 

\tanx\ = V^J/r . (11) 

The zero-energy radial observer sees the observed photon energy Eohs, equation (jlOp . 
relative to its energy at rest at infinity to be 



l-BJ^/r^ _ IsecxI 

-B ~ ■ ^^^^ 



4. Example: inside a Schwarzschild black hole 



Black holes are extreme examples of curved spacetime, perfect for exploring with 
computer visualization [9) [TOl [11] . 
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Figure 4. Six frames from a visualization [5] of tlie scene seen by an observer 
falling into a Schwarzschild black hole on a geodesic with specific energy and 
angular momentum E = 1 and L = 3.92M geometric units. From left to right 
and top to bottom, the observer is at radii of 6. OHM, 3. DOOM, 1.997M, 1.613Af, 
0.786M, and 0.045M geometric units. The illusory horizon is painted with a dark 
red grid, while the true horizon is painted with a grid colored with an appropriately 
red- or blue-shifted blackbody color. The schematic map at the lower left of 
each frame shows the trajectory (white line) of the observer through regions of 
stable circular orbits (green), unstable circular orbits (yellow), no circular orbits 
(orange), the horizon (red line), and inside the horizon (red). The clock at the 
lower right of each frame shows the proper time left to hit the singularity, in 
seconds, scaled to a black hole mass of 5 X 10^ Mq, similar to the mass of the 
Milky Way's supermassive black hole |12II13I . The background is Axel Mellinger's 
Milky Way [1] (with permission). 
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4.1. Visualization 

Figure S] shows six frames from a visualization [Sj of the scene seen by an observer 
falling into a Schwarzschild black hole. The observer starts from outside the horizon, 
falls on a free-fall trajectory, passes through the horizon, and terminates at the central 
singularity. 

Figure [5] shows frames three to six of Figure |4] in stereo. In these stereo views, 
the distance from any point on the scene to the observer has been set equal to the 
observed afline distance Aobs ■ An animated version of the stereo visualization is at [S] . 

The field of view in Figures |4] and [5] is 105° across the diagonal, with a 4 x 3 aspect 
(84° X 63°) in the normal scene, and a square aspect (74.2° x 74.2°) in the stereo 
scene. In the stereo scene, the eyes are 0.04M geometric units (0.02 Schwarzschild 
radii) apart. The background is Axel Mellinger's Milky Way [4], reproduced here with 
permission. The view direction is such as to put the radial axis between the observer 
and the black hole's center 6.32° away from the center of the frame, in a direction 
rotated 6.4° clockwise from downward on the frame (corresponding to a position of 
approximately 6.31 on a clockface). Despite the appearance of later images of Figure|4] 
that give the impression that the observer is looking sideways, in fact the observer is 
always looking in the same direction, approximately towards the black hole's center. 
The deceptive appearance is caused by special relativistic aberration. 

The observer is falling on a geodesic with specific energy and angular momentum 
E = 1 and L — 3.92M geometric units (c = G = 1). The angular momentum is slightly 
less than the angular momentum, L = AM , that would put the observer on the E = 1 
unstable circular orbit. In the complete animated visualization, the observer's orbit 
starts at 2060M geometric units from the black hole, in the plane of the Galaxy, with 
the black hole positioned directly towards the Galactic Center. The orbit is inclined 
at 5.7° to the Galactic plane. The grid on the black hole has its north pole tilted 
by 16° relative to Galactic North Pole, towards Galactic longitude I = 322.1°. The 
Figure shows the view from radii 6.011M (just outside the innermost stable circular 
orbit at r = 6M), 3.000Af (at the photon sphere at r = 3M), 1.997M (just inside the 
horizon at r = 2M), 1.613M, 0.786M, and 0.045M (near the singularity at r = 0) 
geometric units. 

The apparent behavior of the horizon, which appears to split into two when 
an observer falls through it, is quite unexpected. A Penrose diagram. Figure [6l 
helps to navigate and make sense of the journey. This diagram shows only two of 
our universe's four dimensions, one time dimension (vertically) and one radial space 
dimension (horizontally). A Penrose diagram is drawn so that light always moves at 
45° from vertical. An observer necessarily moves at less than the speed of light, so 
follows an upward worldline that is always less than 45° from vertical. 

As its Penrose diagram. Figure [6l shows, the Schwarzschild geometry contains 
both a true horizon, and something that we call the illusory horizon. In the analytically 
extended Schwarzschild geometry, the illusory horizon is an actual horizon, the horizon 
of a white hole and a parallel universe, sometimes also called an antihorizon, or past 
horizoE|§|. In a real black hole formed from the collapse of the core of a star, there is 
no white hole horizon. Rather, the horizon is replaced by an exponentially redshifting 
region that contains the collapsing star and its interior (and beyond). As time goes by, 
the light escaping from the collapsing star becomes exponentially redshifted, giving 

§ We choose not to use the terms past and future horizon because a future horizon becomes a past 
horizon after you have passed through it. 
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Figure 5. Stereo versions of frames three to six of Figure [J] with the observer 
at radii of l.ggTM, 1.613M, 0.786M, and 0.045M geometric units. To view, cross 
your eyes, and relax your focus. The horizon grids should pop out, while the rest 
of the scene remains in the background. 
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Figure 6. Penrose diagram of a Schwarzsehild black hole. The arrowed 
line represents the worldline of an infaller. The wiggly lines represent ingoing 
(blue) and outgoing (red) photons emitted from the true and illusory horizons. 
In the analytically extended Schwarzsehild geometry, the illusory horizon is a 
true horizon, the horizon of a white hole and parallel universe. In a real black 
hole formed from the collapse of a star, the illusory horizon is replaced by the 
exponentially redshifting and dimming surface of the collapsing star. As time 
goes by, the exponentially dimming surface becomes indistinguishable from the 
white hole horizon of the extended Schwarzsehild geometry. In Figures [4] and 
\5\ the black disk painted with a dark red grid is the illusory horizon, while the 
whitish grid is the true horizon. 

the illusory horizon its black appearance. In this paper, we suppose that the black 
hole collapsed long enough ago that it has become effectively indistinguishable from 
a Schwarzsehild black hole, and we refer to the exponentially redshifting surface as 
the illusory horizon. See [H] for animations that depict the idealized collapse of a 
spherical star to a black hole, and spacetime and Penrose diagrams thereof. 

When an observer outside the horizon of a real black hole observes the horizon, 
they are actually observing the illusory horizon, the surface of the collapsed star, 
exponentially dimmed to blackness. 

The Penrose diagram, Figure |6l shows that when the observer subsequently falls 
through the horizon, they do not fall through the horizon they see, the illusory horizon. 
Rather, they fall through the true horizon, which is not visible to them until after they 
have passed through it. Once inside the horizon, the infaller sees both true and illusory 
horizons. The illusory horizon appears generally ahead of the infaller, in the direction 
towards the black hole, just as it did when the infaller was outside the horizon. The 
true horizon appears generally behind the infaller, in the direction away from the black 
hole. 

Inside the horizon, all geodesies, including those of light rays, necessarily fall to 
smaller radii. It is consistent to conceive that space is falling superluminally (faster 
than light) inside the horizon [15]. Geodesies inside the horizon can be classified as 
"ingoing" or "outgoing" depending on whether they are moving backward or forward 
in Schwarzsehild coordinate time t, that is, on whether they have positive or negative 
energji]]. In the Penrose diagram. Figure [U radial light rays moving to the left inside 
the horizon are ingoing (positive energy, going with the inflow of space) , while radial 

II Inside the horizon of a Schwarzsehild black hole, positive energy (ingoing) geodesies go backwards 
in conventional Schwarzsehild coordinate time t. The sign comes from imposing t — > oo at the true 
horizon, and t — > ~oo at the illusory horizon. 
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light rays moving to the right are outgoing (negative energy, going against the inflow) . 
Outside the horizon all geodesies move forward in Schwarzschild coordinate time t, 
and can be classified as infalling or outfalling depending on whether they are moving 
to smaller or larger radii. For brevity, we use the terms ingoing and outgoing to refer 
also to infalling and outfalling geodesies outside the horizon. 

From outside the horizon of the black hole, first two frames of Figure 21 the 
observer sees only the illusory horizon. The illusory horizon in Figure |4] is painted 
with a dark red grid, as befits the fact that objects at this horizon appear (disappear?) 
infinitely redshifted. The illusory horizon appears to recede into the infinite distance, 
as photons emanating from the horizon are able to orbit the black hole an infinite 
number of times near the photon sphere before peeling off into the observer's eye. 

In frame two of Figure |31 the observer is at the radius r = 3M of the photon 
sphere, where light orbits in circles around the black hole. The circular orbit is 
unstable, and an observer at this radius does not see light concentrate at the photon 
sphere. 

Inside the horizon, frames three to six of Figure lU the observer sees both illusory 
and true horizons. The illusory horizon remains infinitely redshifted, even though the 
observer is inside the horizon. By contrast, an object falling through the true horizon 
appears redshifted or blueshifted by a finite (not infinite) amount. The true horizon 
in Figure m is painted with a grid in a blackbody color that the observer would see an 
object at the horizon to have if its emitting temperature were 4096 K (peach white) 
and it were free- falling radially from zero velocity at infinity {E = 1, L — 0). 

In frame three of Figure |4j the observer is just inside the horizon. At the instant 
that the infaller passes through the true horizon, the true horizon appears not as a 
surface, but rather as a line (which appears in projection to the observer as a point) 
that extends from the position of the observer down to the illusory horizon. 

It is sometimes asserted that an observer near the horizon sees the outside universe 
concentrated into a tiny, highly blueshifted, circular patch of sky directly above them. 
This would be true if the observer were at rest in Schwarzschild coordinates, but this 
is a highly unnatural situation, requiring the observer to accelerate enormously just to 
remain at rest. At and inside the horizon, it is impossible for an observer to remain at 
rest, since space is falling at or faster than the speed of light [15]. The more realistic 
situation is that the observer falls, as in the case illustrated by Figures |4] and [H 

In frames three to six of FigurelH the observer is inside the horizon. Together, the 
true and illusory horizons form apparently 2D surfaces that encompass the infaller, 
the "Schwarzschild bubble." The illusory and true horizons join at a circle, where the 
photon angular momentum J is infinite. More distant parts of the illusory horizon are 
visible through the true horizon. Light from the distant illusory horizon was emitted 
upward, but turned around and fell back through the true horizon. 

As the observer falls inward, the Schwarzschild bubble expands horizontally. In 
the limit as the observer approaches the singularity, the edge of the Schwarzschild 
bubble extends to an angle of tt around the black hole, so that the bubble finally 
encompasses the full surface of each of the true and illusory horizons. 

As the observer approaches the singularity, the horizons appear to flatten out, 
and appear to approach each other ever more closely, as illustrated in the final frame 
of Figure S] The appearance is caused by the growing tidal field near the singularity, 
which aberrates the view so as to shift the apparent positions of objects away from 
vertical towards horizontal. 
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Figure 7. Cross-sections through the effective 3-dimensional scene perceived 
by an observer who free-falls radially into the Schwarzschild black hole from zero 
velocity at infinity, with E = 1 and L = 0. The six successive views show the 
location of the true (blue) and illusory (red) horizons From top to bottom, the 
observer is at radii of 3M, 1.999M, IM, 0.5M, O.IM, and O.OIM geometric units. 
The small circle, diameter IM geometric unit, at the center of each frame marks 
the position of the observer. The third-from-top frame, where the observer is at 
IM geometric unit, shows additional detail: the various lines show the perceived 
location of emitting surfaces at, from top to bottom, radii of 3.5M, 3M (photon 
sphere), 2.5M, 2M (horizon), and 1.5M geometric units. Tick marks on the 
horizons are spaced every 30°. The lines are blue where the emitted photon is 
initially ingoing, and red where the emitted photon is initially outgoing. 

4-2. Scenes are constructed by Lorentz transforming out of radially Jailing frames 

Scenes seen by observers at the same position but with different motions are 
related by a Lorentz transformation, in the manner illustrated by Figure O It is 
advantageous to construct a scene in a frame where symmetries are manifest, and then 
to Lorentz transform the scene to the frame of the moving observer. For example, the 
visualizations in Figures H] and [S] were constructed by Lorentz transforming scenes 
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Figure 8. Stereo version of the scene seen by an observer free-falling radially 
from zero velocity at infinity, with specific energy E = 1 and angular momentum 
L = 0. The observer is at a radius of IM geometric unit, as in the third-from-top 
frame of Figure [7] The observer is looking in a direction 30° below horizontal. 
The grid above is the true horizon, while the grid below is the illusory horizon. 



calculated from the perspective of a radially falling observer, as described in the 
remainder of this subsection. 

The spacetime of a spherical black hole has rotational symmetry about the radial 
axis joing the center of the black hole and the observer. Ray-tracing is simplified and 
accelerated in a frame where the observer's motion is along the radial axis, since the 
rotational symmetry about the axis remains manifest in such a frame. 

The rotational symmetry holds for any radial motion of the observer. A natural 
choice of radial frame is that of an observer who free-falls radially from zero velocity 
at infinity, with specific energy E — 1 and angular momentum L = 0. The radial free- 
fall frame exists at all radii, both outside and inside the horizon. Another possible 
choice of radial frame is the rest frame, where the observer is at rest in Schwarzschild 
coordinates. The rest frame fails at and inside the horizon, since space is falling 
superluminally inside the horizon, so no physical object can remain at rest. A third 
choice of radial frame is the zero energy frame, E = L = 0, which exists only inside 
the horizon. 

Figure [7] illustrates cross-sections through the effective 3-dimensional scene 
perceived by an observer who free-falls radially from zero velocity at infinity, with 
E — 1, L = 0. The Figure shows six successive views of the location of the true 
and illusory horizons of the Schwarzschild black hole. The radial axis joining the 
black hole's center and the observer lies along the vertical mid-line of the Figure. The 
geometry is rotationally symmetric about the radial axis. 

The third frame from top in Figure [7] shows additional detail for the observer at 
a radius of IM geometric unit. The Figure shows the perceived location of emitting 
surfaces at radii of 3.5M, 3M, 2.5M, 2M, and 1.5M geometric units. Inside the 
horizon, an observer can see points only at radii larger than their own: points at 
smaller radii are in the future direction, and therefore invisible. The observer sees the 
surface at 1.5M geometric units to surround them. Ingoing photons (blue) appear 
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Figure 9. Cross-sections through the effective 3-dimensional scene, similar to 
Figurc[7l but as perceived by an observer who is infalling on the zero-energy radial 
geodesic, which makes manifest the symmetry between ingoing and outgoing. 
The zero-energy geodesic exists only inside the horizon, so only images where the 
observer is inside the horizon can be shown. From top to bottom, the observer 
U at radii of 1.999Af, IM, O.Sfly, O.IM) and O.OIA/ gcometfifr^a nits, as in all but 
the top frame of Figure [7] The second-from-top panel, where the observer is at 
lA/ geometric unit, shows additional detail similar to the third-from-top panel of 
Figure [7] the various lines show the perceived location of emitting surfaces at, 
from top to bottom, radii of 3.5M, 3M (photon sphere), 2.5M, 2M (horizon), and 
1.5M geometric units. The lines are blue where the emitted photon is initially 
ingoing, and red where the emitted photon is initially outgoing. 

generally above the observer (away from the black hole), while outgoing photons (red) 
appear generally ahead of the observer (towards the black hole). 

Figure [8] shows a stereo visualization of the view depicted in the third frame 
from top in Figure [T] The observer is at a radius of IM geometric unit, inside the 
horizon. The view shares some of the parameters of the scenes in Figures [3] and E) 
the field of view is 105° across the diagonal, the orientation of the grid on the horizon 
is the same, and the black hole is positioned directly in front of the Galactic Center, 
which was the starting point of the trajectory in Figures 2] and [51 However, the view 
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Figure 10. Penrose diagram of the Schwarzschild black hole, with the vertical 
arrowed line showing the worldline of an infaller who falls on the zero-energy radial 
geodesic. The wiggly lines represent ingoing (blue) and outgoing (red) photons 
emitted from the true and illusory horizons. 




Figure 11. Stereo version of the scene seen by an observer free- falling radially on 
the zero energy geodesic, E = L = 0. The observer is at a radius of lAI geometric 
unit, as in the second- from- top frame of Figure |9] The observer is looking in 
the horizontal direction. The scene is the same as in Figure [H] Lorentz-boosted 
radially upward. 



direction differs from Figures U and [SJ the observer is looking in a direction 30° below 
horizontal, rotated left-handedly (scene shifts to the right) by 54° from Galactic North 
Pole about the radial (vertical) direction. 

The view from inside a black hole is most symmetrical in the frame of an observer 
radially free-falling on the zero-energy geodesic, E = L = 0, at the border between 
ingoing {E > 0) and outgoing {E < 0). Figure ^ shows cross-sections through 
the effective 3-dimensional scene perceived by an observer on the radial zero-energy 
geodesic. This is essentially the same set of views as Figure[7l but from the perspective 
of the zero-energy geodesic. The scenes in Figures [7] and IH] are related by a radial 
Lorentz boost. The zero-energy frame reveals the symmetry between ingoing and 
outgoing geodesies, evident in Figure [S] as a reflection symmetry, about a horizontal 
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plane passing through the observer, between the perceived locations of ingoing (blue) 
and outgoing (red) surfaces. 

Figure [TU] shows the Penrose diagram of the black hole with the observer falling 
on the zero-energy radial geodesic. Figure [Til shows the scene seen by such an observer 
at a radius of IM geometric unit, the same as illustrated in the second panel from 
top of Figure El The scene in Figure [TT] is the same as in Figure [3 Lorentz-boosted 
radially upward. The observer in Figure [Tl] is looking in the horizontal direction. 
The true and illusory horizons show reflection symmetry about the midplane of the 
view. The symmetry does not extend to the coloration of the horizons, since objects 
are falling towards the observer through the true horizon above, and away from the 
observer through the illusory horizon below. 



5. Perceptual distances 

The affine distance may be the natural measure of distance along a null geodesic in 
general relativity, but this distance is not directly measurable to an observer using only 
local measurements. In this section we therefore consider various perceptual distances 
that are directly measurable by the observer. We use the example of Section |4] to 
compare perceptual distances to the afhne distance. Figure [131 



5.1. Size distance 

The perceived angular size of an object of known proper size provides one measure of 
distance, known to astronomers as the angular diameter distance. The size distance 
requires prior knowledge of the object's actual size. 

emitter 




Figure 12. The size distance Lsizc follows from the apparent angular size 5a of 
an object of known proper size 51. 



The size distance Lsizo may be calculated by following a narrow bundle of light 
rays along the past lightcone of the observer from the observer to the emitter, as 
illustrated in Figure [121 The bundle is characterized by its cross-section in the 2- 
dimensional spatial plane that lies in the 3-dimensional past lightcone of the observer, 
and is orthogonal to the wavevector A:^. The proper width of the bundle at any point 
along it is SI = \/Sl'^6l„, where Sl"^ is an interval across the 2-dimensional cross-section. 
The bundle starts at a point at the observer, spreading out with angle Sa 

DSl , , 

(13) 

where D/D\ denotes covariant differentiation with respect to affine distance A. As 
remarked at the end of the introduction to SJ31 the affine distance here is understood to 
be normalized to the rest frame of the observer, so that it measures proper distances 



Sa ~ „ , 
DX 
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Figure 13. Perceived reciprocal size distance l/Lsi^g. (upper panel) and 
binocular distance 1/Lbin (lower panel) to the true (blue) and illusory (red) 
horizons of a Schwarzschild black hole perceived by an observer inside the black 
hole, as a function of the viewing angle x relative to the vertical axis, Figure [S] 
with 0° being up to the sky above, and 180° down to the black hole below. The 
observer is at a radius of 1 geometric unit, and is infalling on the zero-energy radial 
geodesic, the same as illustrated in Figure [TTI and the second- from- top panel of 
Figure[9l Dashed and dotted lines show the reciprocal distances respectively in the 
polar and azimuthal directions (per the diagram at right), while thick solid lines 
show the average of the polar and azimuthal reciprocal distances. The averaged 
reciprocal distances provide a good estimate of the afflne distance (thin solid black 
lines) for viewing angles not too far off axis. 



in the local frame of the observer. The size distance Lsizc is the ratio of the width 
Slcm of the light bundle when it arrives at the emitter to the angle 6a subtended by 
the bundle at the observer: 

Lsizo = ^ . (14) 
oa 

The size distance can be calculated by integrating the equation of geodesic deviation 
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along the null ray from observer to emitter, 

= R.x,''k'^k'^Sl^ , (15) 

subject to the initial conditions that the spatial separation SI''' lies in the light 
cone, is orthogonal to the wavevector, k^Sl'^ — 0, and is initially zero but with 
non-zero infinitesimal derivative DSl" /DX, equation ([T^. The equation of geodesic 
deviation (|15p. which depends on the Riemann curvature tensor RkX^iv, is a linear 
equation for the evolution of deviations Sl'^ in the two-dimensional spatial plane 
transverse to the wavevector fc^. The transformation matrix RKXfii^k'^^k'^ is symmetric 
in Ai/, and the solution therefore has two orthogonal eigenvectors. There are 
two corresponding size distances in orthogonal directions. Rather than integrate 
equation (I15|) directly, we prefer to evaluate Sl'^ by ray-tracing along the null ray, 

Sr ^6 J dx" J k^dX , (16) 

subject to the constraint that the observed affine distance from observer to emitter is 
constant, S (£'obs / dA) = 0: 

^1"= fs( -^VobsdA 



Sk" dA - A In Sobs J k^dX . (17) 

The second term on the right hand side of equation (jl7p . involving the variation 
6 In i?obs of the observed photon energy in different directions along the ray bundle, 
appears because in the Schwarzschild geometry it is convenient to do the ray-tracing 
with the wavevector k'^ = dx'^ /dX and affine distance A normalized to observers at rest 
at infinity, equations (jS]) and ([T]) (equation (jl6p is valid regardless of normalization). 
In this case the observed affine distance is related to the rest-at-infinity affine distance 
by equation ([5]), with observed energy E'obs from equation pUj) . 

The upper panel of Figure [13] shows the reciprocal of the polar and azimuthal 
size distances to the true and illusory horizons of the Schwarzschild black hole as seen 
by the observer whose view is illustrated in Figure [TT] and in the second panel from 
top of Figure (S) The two size distances differ, but the average of the inverse size 
distances provides a good estimate of the affine distance, at least as long as the two 
size distances are not too discrepant. 



5.2. Binocular distance 

The size distance has the drawback that the observer must know in advance the 
actual size of the object being observed. The size distance may be imprecise, or even 
completely wrong, if the actual size fails to conform to the observer's expectation. 

A measure of distance used successfully by earthly animals is the binocular 
distance, also known to astronomers as parallax distance. The binocular distance 
is the radius of curvature of the 2-dimensional wavefront of light rays emitted by 
the emitter, as measured by the observer in their own locally inertial rest frame, as 
illustrated in Figure 1141 Binocular distance does not require prior knowledge of an 
object's actual size. A different definition of binocular distance has been proposed 
by |16| . but that definition does not yield a consistent measure of distance in fiat 
(Minkowski) space (see [Appendix B[ ). 
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observer 

emitter k + 




Figure 14. Tlie binocular distance Lbin is tiie radius of curvature of a wavefront 
of light emitted by an emitter and observed by an observer. 



Question for the reader: if you look at this page through a magnifying glass, does 
your binocular vision perceive the text to be closer or farther away? Try it. You can 
anticipate the answer by looking at Figure [141 

Consider two eyes a small distance Sl'^ apart. The covariant difference dk^ in the 
photon wavevectors perceived by the two eyes is 

Sk'' = SFkf",^ , (18) 

where the colon ; represents covariant differentiation. Figure [T3] illustrates the two 
wavevectors k^ and fc'' + Sk^^ seen by the two eyes. Because the wavevector k^^, 
equation 1^, is normalized to unit energy, the difference Sk = yJ5kf^6k^ of the photon 
wavevectors is just equal to the angle between the two wavevectors. The resulting 
binocular distance Lbin is 

Lbi„ = ^ . (19) 

The eye separation 51'^ is confined to lie on the 3-dimensional future lightcone of the 
emitter, and the geodesic equation Dk'^/DX = k^k^-^i, = implies that 5k^^ vanishes 
for 51'^ parallel to the wavevector k'^ . Thus the covariant difference 5k^ depends only 
on the components of the eye separation in the 2-dimensional spatial subsurface of the 
emitter lightcone that is orthogonal to the wavevector, and without loss of generality 
the eyes can be taken to lie in this 2-dimensional subsurface. More explicitly, the 
covariant difference in photon wavevectors between the two eyes is 

f dk^^ \ 

sk^ = 5i''[^-^+n^k-j ' (^'^^ 

in which the first term, 51^ dk^ / dx^ , represents the change in the photon wavevector k^ 
between the two eyes, while the second term, Sl'^T'^^k'^ , depending on the connection 
coefficients T^^^, represents the effect of parallel-transporting the photon wavevector 
from one eye to the other. The second term corrects for the difference in spacetime 
frames of the two eyes that results from the arbitrariness of coordinates in general 
relativity. It is only the covariant difference in photon wavevectors that is physically 
measurable. Evaluating the first term on the right hand side of equation (|^D)) involves 
ray-tracing from emitter to observer, while the second term on the right depends only 
on quantities local to the observer. 

In geometric optics pT, §22.5], the future lightcone of the emitter is a surface of 
constant phase V', the wavefront. The photon wavevector fc^ is given by the gradient 
of the phase, /c^ = = dtp/dx'^. The null condition fc^fc^ = implies that the 
photon wavevector k'^ is orthogonal to the gradient /c^ of the phase, and therefore lies 
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in the lightcone, the surface of constant phase. The covariant derivative of the photon 
wavevector k^^.^ is the curvature of the phase, a symmetric matrix: 

kf^-u = ■ (21) 

In the locahy inertial frame of the observer, and on the hghtcone, the non-vanishing 
components of the curvature matrix ^-^^i, constitute a 2 x 2 symmetric matrix in the 
spatial directions orthogonal to the wavevector. The eigenvalues of the symmetric 
matrix il^-fn, are the reciprocal radii of curvature 1/-^|| and l/L± in two perpendicular 
directions. In a frame aligned with the eigenvectors, the covariant difference Sk'^, 
equation (|18p . in the photon wavevectors seen by two eyes becomes 

- ^ ^-Sl( " , ° ) . (22) 



As with the size distance equation (jl7p . the factor of -Eobs is introduced into 
equation (j22p because in the Schwarzschild geometry it is convenient to do the ray- 
tracing demanded by equation (l20l) with the wavevector k^^ normalized to rest-at- 
infinity observers, equation ([6]). The wavevector must then be rescaled to the actual 
observer with the observed energy i^obs from equation (fTO| . In the Schwarzschild 
geometry, the two radii of curvature L|| and L± of the wavefront correspond to those 
with eyes separated in the polar and azimuthal directions, as indicated in the diagram 
to the right side of Figure [TS] 

In fiat spacetime (Minkowski space), the wavefront from a point emitter is 
spatially spherical, and the radius of curvature L is the same, and equal to the affine 
distance, regardless of how the eyes are oriented about the line of sight. 

In curved spacetime, the curvature matrix has two different eigenvalues, so the 
wavefront is ellipsoidal, yielding two different measures of binocular distance L|| and 
L±. The confiicting visual cues would disorient an animal with binocular vision, just 
as conflicting visual and motion cues disorient people inside immersive environments 
such as airplanes. 

The lower panel of Figure [13] shows the reciprocal of the polar and azimuthal 
binocular distances to the true and illusory horizons of a Schwarzschild black hole, as 
seen by the observer whose view is shown in in Figure [TT] and in the second-from-top 
panel of Figure IH) The observer is inside the horizon at a radius of IM geometric 
unit, and is radially infalling on the zero-energy geodesic. On-axis (0° and 180°), the 
binocular distances agree with the affine distance, but off-axis the polar and azimuthal 
binocular distances disagree, a symptom of the fact that the wavefront is ellipsoidal 
rather than spherical. 

The observer can also deduce an absolute distance from the fractional difference 
in size distances perceived by two eyes separated along the line of sight. However, 
this distance is simply the binocular distance, and so does not provide an additional 
measure of distance. 



5.3. Trinocular distance 

One practical solution to the problem of depth perception in curved space would 
be to evolve three eyes in a triangular (non-collinear) pattern, such as illustrated in 
Figure [m Three eyes would allow an animal to process the ellipsoidal wavefront into 
a best distance, one that would improve their chance of survival. 

The lower panel of Figure [T3] shows that the mean of the reciprocal polar and 
azimuthal binocular distances agrees well with the affine distance even well off-axis 



Stereoscopic visualization in curved spacetime: seeing deep inside a black hole 



22 




Figure 15. An imaginative artistic rendering of a three-eyed inhabitant of a 
highly curved spacetime. The triangular configuration of eyes would be suitable 
for trinocular vision. Drawing by Wildrose Hamilton. 



where the wavefront is quite non-spherical. This suggests that a good strategy for 
the brains of three-eyed animals would be to infer a trinocular distance Ltri from the 
mean of the reciprocal binocular distances, that is, from the trace of the wavefront 
curvature matrix: 

1/Lt,, - i (1/^11 + ■ (23) 

The agreement of the trinocular distance Ltri, equation (|23p. with the affine 
distance can be regarded as a consequence of the Raychaudhuri equation [T71 fT5] , 
which shows that in empty space (zero energy- momentum tensor), the expansion of 
a bundle of light rays depends only quadratically on the shear. Thus the expansion 
is unaffected to linear order by the shear. In other words, the expansion is more or 
less what it would be in the absence of shear, provided that the ellipticity of the ray 
bundle is less than unity. 

The trinocular distance fails as an estimate of affine distance when the ellipticity of 
the ray bundle exceeds unity. The wise animal would learn not to trust the trinocular 
distance if the two binocular distances are too discrepant. 

As a brief reminder of where the Raychaudhuri and related equations come from, 
a bundle of light rays may be characterized by the four Sachs scalars, the expansion, 
the rotation (or vorticity), and complex shear [T71 §9], [TE]. The Sachs scalars are 
equivalent to the 2x2 matrix of tetrad-frame connection coefhcents orthogonal to the 
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wavevector evaluated in a sequence of locally inertial frames parallel-transported along 
the null ray. The evolution of the Sachs scalars is governed by the usual equations 
relating the tetrad-frame connection coefficients to the Riemann tensor. In geometric 
optics, where the wavevector is the gradient of a scalar, the rotation vanishes. If the 
tidal stress (spin-2 component of the Weyl tensor) along the bundle is non- vanishing, 
then shear accumulates along the bundle, causing the wavefront to become ellipsoidal. 
The equation for the expansion, the Raychaudhuri equation, depends quadratically 
on the shear. 

6. Conclusion 

A scene in any arbitrary curved spacetime may be rendered consistently in 3D 
by placing objects at their observed angular positions and at their observed afhne 
distances. Such a scene may be displayed to an audience in stereo in the usual fashion, 
by projecting two views of the scene, as if it were in flat space, as seen by two cameras 
a small distance apart. 

In flat (Minkowksi) spacetime, affine distance coincides with proper distance 
measured in the rest frame of the observer, which in turn coincides with the binocular 
distance perceived by a two-eyed observer, even if the observer is moving through the 
scene at near the speed of light. 

In curved spacetime, however, the 3D stereo scene obtained by placing objects 
at their affine distances is not the same as the scene that an observer with binocular 
vision would perceive if they were actually present in the curved spacetime. 

The fact that binocular vision fails in curved spacetime is not a fundamental 
obstacle to 3D perception. Rather, it is simply a limitation of beings who have evolved 
in flat spacetime. In flat spacetime, wavefronts of light from an emitter propagate 
outward spherically, and binocular vision, which measures the radius of curvature of 
the wavefront, yields a unique estimate of distance that coincides with affine distance. 
But in curved spacetime, tidal forces distort a wavefront from sphericity, causing 
it to become locally ellipsoidal, with two distinct curvature eigenvalues, hence two 
conflicting measures of binocular distance. Three eyes. Figure [151 would allow an 
animal to process the ellipsoidal wavefront into an improved estimate, the trinocular 
distance. It follows from the Raychaudhuri equation that the average of the two 
inverse radii of curvature would provide a trinocular distance that is a good estimate 
of the affine distance provided that the ellipticity of the wavefront is not much greater 
than one. 
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Appendix A. Rigid bodies in curved spacetimes 

It is impossible to place a flat piece of paper on the surface of a sphere without 
wrinkling the paper. For essentially the same reason, there is no such thing as a 
rigid body in curved spacetime. The difficulty arises only if the size of the body 
is appreciable compared to the length scale of curvature of the spacetime. General 
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relativity postulates the existence of locally inertial frames, and if the body is small 
enough, then it will fit in a locally inertial frame with negligible distortion. 

In the scene shown in Figure [U the Earth is placed artificially, as if it were a 
rigid body, near a Schwarzchild black hole. This Appendix describes the convention 
adopted here for placing a large rigid body in Schwarzschild spacetime. 

The GuUstrand-Painleve [El [2Ql [15] version of the Schwarzschild line-element for 
a spherical black hole of mass M is 

ds^ ^-dtl + {dr -vdtsf +r^{de^ +sm^ed4)^) , (A.l) 

where tg is the free-fall time, the proper time attached to observers who free-fall 
from zero velocity and angular momentum at infinity, and v is the Newtonian escape 
velocity 




(A.2) 



The Gullstrand-Painleve line-element (jA.l|) has the property that it is spatially flat 
at constant free-fall time dtg = 0. In a neighborhood of any point, Gullstrand- 
Painleve coordinates define a set of locally inertial coordinates attached to radial 
free-fall observers. Thus a consistent, albeit somewhat arbitrary, approach is to 
treat Gullstrand-Painleve coordinates as defining a mapping between Schwarzschild 
and Minkowski spacetime; that is, the coordinates {iff , r, 0, </>} are treated as polar 
coordinates of Minkowski space. For example, in the simple case of a body whose 
center-of-mass is at rest in the frame of a Gullstrand-Painleve free-faller, the body 
would be placed in the flat Gullstrand-Painleve spatial hypersurface as if the space 
were plain 3-dimensional Euclidean space. In the case shown in Figure [l] the Earth 
is in orbit around the Schwarzschild black hole, so its frame is boosted relative to 
Gullstrand-Painleve free-fallers. 

Appendix B. Alternative definition of binocular distance 

In this paper, the binocular distance from an emitting source has been taken to be the 
radius of curvature of the 2-dimensional wavefront from the source, as perceived by 
an observer in their own locally inertial frame. In flat (Minkowski) space, this yields 
a consistent measure of distance regardless of the motion of the emitter or observer. 

Chapter 3 of [16] considers an alternative definition in which a binocular scene is 
constructed from two images viewed simultaneously in the rest frame of the observer. 
By contrast, the definition in the present paper takes the two images to lie in the 
lightcone of the emitter, in which case there is a slight time delay between the two 
images, equal to the differential travel time of the light to the two eyes. The definitions 
of [12] and the present paper agree when the eyes are looking straight ahead, so that 
there is no differential time delay in the light travel time to the eyes, but differ when 
the eyes are not looking directly ahead. 

As [16] points out, two simultaneous images do not yield a consistent stereo view. 
By contrast, the present definition of binocular distance as the radius of curvature of 
the wavefront yields a consistent measure of distance, and thus a consistent stereo view, 
in Minkowski space. Successful binocular vision would compensate for the differential 
light travel time between the two eyes. Such compensation would be quite feasible, 
since it depends only on the distance between the eyes and the observed angle of the 
emitter relative to directly ahead. 
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Human vision is too slow to distinguish images separated by a light travel time 
between the eyes, which is less than a nanosecond. More precise vision is needed to 
achieve successful binocular vision in a relativistic setting. 

Astronomers are familiar with the fact that precision location of objects on the sky 
(astrometry) requires faithful tracking of the wavefront. For example, in Very Long 
Baseline Interferometry |21j images seen by two or more telescopes are combined with 
an appropriate time delay so that all see the same wavefront. The Keck telescope, 
consisting of twin 10-meter optical/infra-red telescopes on the summit of Mauna Kea 
in Hawai'i, can be operated in interferometric mode [22]. The Keck telescope thus 
realizes the requirements for successful binocular vision in a relativistic environment. 
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